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Preface 


The  primary  purpose  of  this  research  effort  was  to 
identify  factors  that  affect  the  warranty  cost-benefit 
analysis  (CBA)  for  the  Sensor  Fuzed  Weapon  (SFW)  System 
because  a  waiver  to  the  essential  performance  requirement 
(EPR)  warranty  was  under  consideration.  The  study  began 
with  an  investigation  of  the  history  of  several  dormant 
weapon  systems  similar  to  the  SFW  and  concluded  by  comparing 
similarities  in  the  nature  of  the  systems  and  the  cost- 
effectiveness  analyses  performed  for  the  warranties. 

This  study  could  not  have  been  completed  without  the 
assistance  and  insight  of  my  thesis  advisors.  Dr.  Ben 
Williams  and  Mr.  Carroll  Widenhouse,  both  with  extensive 
experience  in  reliability  and  maintainability,  helped  in 
developing  a  structure  for  this  research  effort.  Next,  I 
would  like  to  express  my  appreciation  to  Major  Michael  Urban 
and  Mr.  Edward  Muller,  of  the  Sensor  Fuzed  Weapon  and 
Airfield  Attack  System  Program  Office  at  Eglin  AFB,  for 
sharing  their  knowledge  and  experience  with  me. 

Donna  E.  Kehrt 


ii 


Table  of  Contents 


Page 

Preface .  ii 

List  of  Figures .  v 

List  of  Tables .  vi 

Abstract .  vii 

I.  Introduction  .  1 

General  Issue  .  1 

Background  .  1 

Problem  Statement  and  Research 

Objective  .  2 

Investigative  Questions  .  2 

Definitions  .  3 

Scope  and  Limitations .  3 

Summary  .  3 

II.  Literature  Review  .  5 

Introduction  .  5 

Discussion  of  the  Literature  ....  7 

EPR  Warranty  Background  .  15 

Built-in  System  Redundancy  .  18 

Conclusion .  19 

III.  Methodology .  21 

Introduction  .  21 

Justification  for  Method  Selected  .  .  21 

Methodology .  22 

Support .  23 

Validation  and  Historical  Accuracy 

of  Method .  23 

Summary .  23 

IV.  Findings .  24 

Introduction  .  24 

Findings  for  Research  Question  One  25 

Findings  for  Research  Question  Two  43 

Siunmary  .  49 

V.  Conclusions  and  Recommendations  .  50 

Introduction  .  50 

iii 


Page 


Conclusions .  50 

Recommendations  for  Further 

Research .  54 

Sviramary .  55 

Appendix  A:  SFW  Sampling  Plan .  56 

Appendix  B:  Reliability  Formula  and  Redundancy 

Figure .  57 

Appendix  C:  Comparison  of  Cost-Benefit 

Analysis  Inputs  .  58 

Bibliography  .  59 

Vita .  61 


iv 


List  of  Figures 


Figure 

1.  OEM  Sequential  Sampling  Plan 


List  of  Tables 


Table  Page 

1.  SFW  Program  Quantities  by  Year .  38 


vi 


AFIT/GLM/LSQ/92S-27 


Abstract 

The  purpose  of  this  research  effort  was  to  identify  factors 
that  affect  the  warranty  cost-benefit  analysis  (CBA)  for  the 
Sensor  Fuzed  Weapon  (SFW)  System  since  an  essential 
performance  requirement  (EPR)  warranty  waiver  was  under 
consideration.  Defense  funding  has  become  scarce,  and  the 
limited  dollars  must  be  spent  wisely.  A  CBA  should  provide 
a  relevant  projection  of  cost-effectiveness  based  on  the 
best  available  data.  The  SFW  and  other  dormant  systems 
present  difficulties  in  determining  cost-effectiveness 
because  the  systems  most  probably  will  be  non-operating, 
non-reusable,  and  non-restorable.  Three  recommendations 
were  made,  relevant  to  dormant  weapon  systems,  that 
pert  ined  to  further  cost  analysis  and  review  of  overall 
warranty  effectiveness.  The  SFW’s  EPR  warranty  will  become 
effective  in  1996.  In  the  meantime,  the  program  manager  can 
examine  more  closely  the  feasibility  of  the  EPR  warranty  and 
the  warranty  in  total . 
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REVIEW  OF  FACTORS  IMPACTING 
WARRANTY  COST-BENEFIT  ANALYSIS 
FOR  THE  SENSOR  FUZED  WEAPON  SYSTEM 

I .  Introduction 

General  Issue 

Public  law  mandates  the  use  of  warranties  for  DOD 
weapon  systems  acquisition  (22:1).  Exceptions  to  complying 
with  the  law  are  the  warranty  is  not  cost-effective,  or  the 
warranty  was  not  in  the  best  interest  of  national  defense; 
and  a  waiver  to  the  public  law  should  be  requested  for  the 
exceptions.  The  General  Accounting  Office  (GAO),  the  DOD 
Inspector  General,  and  RAND  Corporation  represent  a  sample 
of  the  organizations  that  have  expressed  concerns  regarding 
the  validity  of  weapon  system  warranty  (WSW)  cost-benefit 
analyses  (CBAs).  Without  an  adequate  CBA,  the  warranty 
purchase  decision  cannot  be  made  with  confidence. 

Background 

Title  10,  Section  2403,  of  the  United  States  Code 
mandates  the  use  of  warranties  for  acquisition  of  DOD  weapon 
systems  with  a  unit  cost  of  more  than  $100,000  or  total 
procurement  cost  in  excess  of  $10  million.  In  1983  Congress 
passed  the  law  to  require  manufacturers  to  provide  weapon 
system  warranties.  Three  types  of  warranties  are  required: 
design  and  manufacturing,  materials  and  workmanship,  and 
performance  specifications.  Congressional  interest  in 
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warranties  remains  high  due  to  the  media  visibility  that 
occurs  when  the  government  procures  a  system  that  does  not 
work  effectively.  The  warranty  requirement  was  intended  to 
prevent  these  procurement  "horror  stories".  Recently, 
however,  review  organizations  have  questioned  whether  some 
warranties  should  have  been  purchased.  The  GAO  issued  a 
report  stating  the  government  spends  hundreds  of  millions  of 
dollars  every  year  on  warranties  and  questions  whether  they 
were  all  cost-effective  (23:3). 

Problem  Statement  and  Research  Objective 

The  specific  problem  was  to:  1)  analyze  the  unique 
attributes  of  the  Sensor  Fuzed  Weapon  (SFW)  System's 
performance  requirements  to  determine  how  they  impact  the 
cost-effectiveness  and  future  desirability  of  a  warranty, 
and  2)  review  SFW  and  other  dormant  system  CBAs  to 
determine  if  a  general  methodology  can  be  developed  to 
improve  future  cost  benefit  analysis  efforts. 

Investigative  Questions 

Two  investigative  questions  were  developed  to  answer 
the  research  objective.  First,  what  system  attributes  make 
the  SFW  different  from  the  typical  warranted  system,  and  how 
do  these  attributes  impact  the  warranty  effectiveness? 
Second,  are  cost  analyses  for  dormant  systems  similar,  and 
would  it  be  possible  to  develop  a  general  CBA  methodology? 
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Definitions 

The  following  terms  used  throughout  the  doctunent  are 
defined  below.  The  definitions  for  EPR  and  WSW  come  from 
APR  70-11,  Acquisition  Management:  Weapon  System  Warranties 
(11:1-2) : 

1.  Cost-Benefit  Analysis  (CBA):  An  analytical 
procedure  to  compare  the  cost  of  the  warranty  to 
the  potential  benefits  derived  from  the  purchase 
of  a  warranty. 

2.  Essential  Performance  Requirement  (EPR): 

Measurable,  verifiable,  trackable,  and  enforceable 
operating  capabilities  and/or  maintenance  and 
reliability  characteristics  of  a  weapon  system 
that  are  determined  necessary  for  the  system  to 
fulfill  the  military  requirement  for  which  the 
system  is  designed. 

3.  Sensor  Fuzed  Weapon  (SFW)  System;  A  munition 
system  in  the  form  of  a  cluster  bomb  unit. 

4.  Weapon  System  Warranty  (WSW):  A  promise  or 
affirmation  given  by  the  contractor  to  the 
government  regarding  the  nature,  usefulness,  or 
condition  of  the  supplies  or  performance  of 
services  furnished  under  the  contract. 


Scope  and  Limitations 

The  research  was  limited  to  the  SFW  and  similar  dormant 
Air  Force  weapon  systems.  The  analysis  performed  may  be 
applicable  to  other  systems  in  the  Air  Force  as  well  as 
other  Defense  Agencies. 


Summary 

The  investigative  questions  are  interrelated  for  the 
following  reason.  The  warranted  performance  requirements 
should  be  selected  based  on  operational  performance 
requirements  for  which  compliance  cannot  be  determined  with 
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certainty  prior  to  or  during  acceptance  testing  (11:4).  The 
cost-benefit  analysis,  therefore,  must  provide  a  relevant 
projection  cf  the  future  based  on  the  stated  performance 
requirements . 

This  chapter  reviewed  the  general  issue,  presented 
background,  stated  the  problem  and  research  objective, 
stated  the  investigative  questions,  provided  definitions, 
and  the  scope  and  limitations  of  the  topic.  Chapter  two 
reviews  the  literature  on  warranties,  chapter  three 
describes  the  methodology,  chapter  four  describes  the 
findings,  and  chapter  five  presents  the  conclusions  drawn 
from  the  findings. 
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II .  Literature  Review 


Introduction 

Topic  Statement  and  Brief  Explanation  of  Kev  Terms. 

The  purpose  of  this  chapter  was  to  examine  literature  on 
weapon  system  warranty  (WSW)  cost-benefit  analyses  (CBAs) 
for  the  Department  of  Defense  (DOD)  and  to  review  EPR 
warranties  and  system  redundancy  as  they  relate  to  the  SFW 
system.  The  Defense  Federal  Acquisition  Regulation 
Supplement  (DEARS)  and  Air  Force  regulations  require  a  CBA 
be  performed  prior  to  the  purchase  of  a  warranty.  The 
program  manager  makes  his  warranty  purchase  decision  based 
upon  the  CBA.  The  decision  may  be  based  on  erroneous 
results  if  the  analysis  is  not  performed  adequately; 
therefore,  CBAs  must  be  accurate. 

Keywords .  The  following  terms  or  acronyms  were  used 
throughout  the  literature  review: 

1.  CBA:  Cost-Benefit  Analysis 

2.  DOD:  Department  of  Defense 

3.  DEARS:  Defense  Federal  Acquisition  Regulation 
Supplement 

4.  FAR:  Federal  Acquisition  Regulation 

5.  GAO:  General  Accounting  Office 

6.  SFW:  Sensor  Fuzed  Weapon 

7.  WSW:  Weapon  System  Warranty 

Justification  of  the  Search  and  Review.  The  government 
may  spend  funds  on  warranties  that  are  not  cost-effective. 
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For  a  major  system,  the  cost  of  a  warranty  can  be  a 
significant  portion  of  total  system  cost.  Therefore,  the 
CBA  is  a  necessary  internal  control  measure.  The  analysis 
must  justify  either  the  purchase  of  a  warranty  or  the  need 
to  obtain  a  waiver  if  the  warranty  is  shown  not  to  be  cost- 
effective.  Several  government  organizations  found 
deficiencies  in  CBAs,  and  they  recommend  improvement  of  the 
analysis  process. 

Scope  and  Limitations  of  Research  Topic.  The  objective 
was  to:  1)  analyze  the  unique  attributes  of  performance 
requirements  for  one-time-use  systems,  specifically  for  the 
Sensor  Fuzed  Weapon  (SFW)  System,  to  determine  how  the 
attributes  impact  cost-effectiveness  and  future  desirability 
for  a  warranty,  and  2)  review  SFW  and  other  dormant  system 
CBAs  to  determine  if  a  general  methodology  can  be  developed 
to  improve  future  cost  benefit  analysis  efforts.  The  topic 
was  limited  to  the  SFW  and  similar  dormant  systems,  although 
applicability  to  other  Department  of  Defense  systems  may  be 
valid. 

Method  of  Treatment  and  Organization.  To  ensure  an 
understanding  of  the  issue,  a  background  of  the  history  of 
Air  Force  warranties  was  accomplished.  The  background  was 
followed  by  collecting  a  list  of  the  current  governing 
regulations  impacting  WSWs.  A  brief  synopsis  of  literature 
regarding  warranty  CBAs  was  presented,  and  was  followed  by 
general  CBA  guidance.  The  CBA  reviews  were  presented 
chronologically  to  illustrate  the  consistency  of  the  problem 
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since  the  inception  of  the  public  law.  Comments  on  the 
status  of  warranties  today  was  given,  followed  by  an 
overview  of  the  cost  analysis  for  the  SFW.  A  short 
discussion  on  Air  Force  warranty  waivers  approved  was 
presented,  and  EPR  warranties  and  the  redundancy  aspects  of 
the  SFW  were  described.  A  summary  was  presented  last. 

Discussion  of  the  Literature 

Background .  The  warranty  provision  was  mandated  by 
public  law  after  Congress  voted  in  1983  to  require 
manufacturers  to  provide  weapon  system  warranties.  The 
requirement  stemmed  from  the  procurement  "horror  stories"  in 
the  early  1980s  (9:574).  Senator  Mark  Andrews,  a  republican 
from  North  Dakota,  initiated  the  effort.  In  defending  the 
proposed  law,  he  argued  that  his  tractor  had  a  warranty,  and 
therefore,  a  weapon  system  should  also  have  a  warranty. 

Much  debate  about  the  issue  occurred  at  the  Armed  Services 
Hearing  in  February  1984.  Several  witnesses  spoke  against 
the  proposed  law.  Secretary  of  Defense  Richard  D.  DeLauer, 
the  chief  Pentagon  witness  stated,  "We  can't  afford  and 
probably  don't  need  to  guarantee  everything"  (9:576).  Air 
Force  Lt  Gen  James  W.  Stansberry  warned  the  panel,  "Once 
modest,  unchallenging  [performance]  goals  are  set,  they  tend 
to  become  self-fulfilling  prophecies"  (9:576).  The  article 
further  explained  that  warranties  do  not  provide  incentives 
to  contractors  to  make  improvements  in  technology  past  what 
is  required  by  the  warranty. 
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Contractors  presented  several  argvunents  against  the 
proposed  law.  First,  warranties  discouraged  innovations  and 
breakthroughs  in  technology  (9:576).  Second,  critical 
differences  existed  between  corranercial  and  military 
equipment,  and  these  differences  were  not  being  considered 
(9:576).  Third,  the  Department  of  Defense,  as  well  as  the 
contractor,  controls  the  design  of  a  system  (9:576). 
Therefore,  the  government  shares  responsibility  with  the 
contractor  in  this  area.  Sen.  Andrews  continued  to  maintain 
his  position  arguing  that  warranties  work  in  the  private 
sector . 

Congressional  interest  remained  high,  and  the  public 
law  was  passed  as  part  of  the  1984  Defense  Appropriation 
Act.  "The  law  requires  that  the  prime  contractor  for  a 
production  weapon  system  provide  written  guarantees, 
starting  with  procurements  after  1  January  1985"  (4:2-3). 

The  three  guarantees  required  were  1)  design  and 
manufacturing,  2)  materials  and  workmanship,  and 
3)  performance  specifications. 

Current  Governance.  The  current  regulations  governing 
weapon  system  warranties  are  listed: 

1.  Federal  Acquisition  Regulation  (FAR), 

Subpart  46.7,  FAC  84-58, 

2.  Defense  Federal  Acquisition  Regulation 
Supplement  (DFARS),  Subpart  46.770-8, 

DAC  88-15, 
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3. 


Air  Force  Regulation  (APR)  173-15,  Cost 
Analysis .  4  Mar  88,  and 
4.  Air  Force  Regulation  (APR)  70-11, 

Acquisition  Management;  Weapon  System 
Warranties .  1  Dec  88. 

The  public  law  is  contained  in  10  United  States  Code  (USC) 
2403.  The  APR  70-11  states  the  requirement  for  a  cost- 
benefit  analysis: 

It  is  DOD  policy  to  obtain  only  cost-effective  WSWs. 
Therefore,  a  CBA  must  be  done  to  determine  whether  the 
contemplated  WSW,  which  will  be  in  the  production 
contract,  is  cost  effective.  A  CBA  must  be  done,  even 
though  the  contractor  may  propose  a  "no-cost”  WSW,  to 
compare  the  government's  cost  of  administering  and 
enforcing  the  WSW  to  the  potential  benefits  to  be 
derived  from  the  proposed  WSW.  (9:4) 

Affordability  must  be  considered  in  relation  to  potential 

benefits  derived  from  the  warranty.  The  CBA  should  indicate 

the  potential  benefits  relative  to  the  expected  cost. 

Synopsis  of  Reports  on  Cost  of  Weapon  System 

Warranties .  A  chronological  synopsis  of  various  reports  and 

articles  concerning  warranty  CBAs  shows  a  continued  concern 

about  the  validity  of  CBA  procedures.  A  synopsis  of  CBA 

guidance  was  obtained  and  is  provided,  and  it  illustrates 

the  nonspecific  nature  of  the  guidance.  This  was  followed 

by  an  article  and  a  report  containing  general  CBA  comments 

regarding  the  current  status  of  warranties. 

CBA  Literature.  The  RAND  Corporation  published  a 

report  in  1987  regarding  weapon  system  warranties.  The 

report  reviewed  implementation  and  theory  of  warranties,  and 
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it  attempted  to  answer  the  question,  "Can  warranties  help  in 
improving  weapon  cost,  schedule,  performance,  and 
reliability"  (21:iii)?  The  authors  were  concerned  that  DOD 
was  spending  funds  on  negotiating,  implementing  and 
enforcing  warranties,  and  DOD  had  no  knowledge  of  the  cost- 
effectiveness.  One  of  the  recommendations  from  the  report 
was: 


.  .  .there  is  a  real  need  for  improved  evaluation 
criteria  and  procedures--and  this  includes 
procedures  for  a  priori  cost-effectiveness 
estimates  as  well  as  ex  post  cost  and  benefit 
evaluations.  So  long  as  those  procedures  are 
limited  to  logistics  or  life-cycle  costs,  it  is 
unlikely  that  any  part  of  the  project  oftxce,  or 
the  user  or  support  communities,  will  be  concerned 
with  more  important  and  more  general  benefits. 

(21:67) 

Procedures  may  need  to  be  improved,  and  a  comprehensive 
review  process  may  need  to  be  implemented.  The  report  also 
stated  concern  regarding  the  lack  of  waiver  requests 
submitted,  and  said  more  policy  is  needed  in  this  area. 

The  RAND  Corporation  published  another  report  on 
warranties  during  early  1989.  It  was  entitled  A  New  View  of 
Weapon  System  Reliability  and  Maintainability.  While  the 
report  was  primarily  directed  towards  a  review  of 
reliability  and  maintainability  weaknesses  in  the  Air  Force 
systems,  it  also  addressed  the  cost  of  warranties.  The 
author  stated,  as  in  the  previous  RAND  report,  the  services 
are  spending  money  without  knowing  if  benefits  will  be 
derived.  He  said,  "...data  and  contractual  language  are 
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being  gathered  sporadically  by  various  agencies..." 

(15:64).  The  report  said  a  process  to  systematically  gather 
and  coordinate  information  and  data  does  not  exist.  The 
warranty  effectiveness  cannot  be  evaluated  if  the  essential 
data  cannot  be  gathered.  The  article  stated  that  there 
should  be  evidence  of  a  connection  between  use  of  warranties 
and  increased  levels  of  reliability  and  maintainability 
prior  to  purchase  of  a  warranty. 

The  United  States  General  Accounting  Office  (GAO) 
published  a  report  to  the  Secretary  of  Defense  on  DOD 
warranties  in  September  1989.  One  purpose  of  the  GAO  review 
was  to  determine  if  cost-effectiveness  analyses  were  being 
performed  as  required  by  both  DOD  and  service  regulations. 
The  review  was  justified  because  hundreds  of  millions  of 
dollars  are  spent  on  warranties  each  year  by  the  services 
(23:2).  The  principal  findings  of  the  report  were  stated  as 
f ol 1 ows : 

Procurement  activities  included  in  GAO's  review 
either  have  not  been  performing  cost-effectiveness 
analyses  or  have  prepared  analyses  that  do  not 
adequately  support  conclusions  that  proposed 
warranties  are  cost  effective.  As  a  result, 
procurement  activities  were  not  considering  waiver 
requests  in  their  decisions  on  proposed  warranties 
because  their  analyses  did  not  provide  a 
convincing  basis  to  support  requests  for  waivers 
in  cases  where  warranties  may  not  have  been 
justified  because  they  would  not  be  cost- 
effective.  (23:3) 


The  GAO  audited  three  procurement  activities  and  discovered 
CBAs  were  either  not  being  performed  or  they  were  not 
prepared  adequately,  and  therefore,  did  not  support  a  valid 
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conclusion  that  the  proposed  warranty  was  cost-effective. 
They  also  stated  the  services  were  not  considering  waivers 
as  an  option  during  the  procurement  process  (23:19).  The 
official  finding  of  the  report  reflected  the  inadequacy  of 
CBAs. 

The  final  report  of  the  System  Acquisition  Management 
Inspection  (SAMI)  of  WSWs  was  published  in  August  of  1990. 
The  team  reviewed  several  aspects  of  warranties,  one  of 
which  was  the  CBA.  The  team  reported  deficiencies  in  the 
data  base  and  review  process  for  CBAs  and  stated,  "as  a 
result  of  this  problem,  many  CBAs  may  be  invalid"  (16:10). 
The  problem  referred  to  the  deficient  data  base.  The  result 
was  an  inefficient  CBA  that  cannot  adequately  support  a 
decision  regarding  cost-effectiveness.  The  team  also 
commented  that  a  good  CBA  process  had  not  been  implemented 
in  any  of  the  organizations  reviewed.  Uncertainty  about  the 
validity  of  a  warranty  CBA  was  the  result  of  a  poor  process. 
The  recommendation  from  the  review  was,  "Devise  and  build  a 
data  base  of  essential  cost  parameters  and  relationships  to 
support  effective  warranty  CBAs"  (16:10). 

Guidance  for  Performing  CBAs.  The  Air  Force 
Conunvini cations  Command  at  Scott  Air  Force  Base  developed  a 
Warranty  Guide  published  in  April  1991.  The  guide  provides 
general  information  on  warranties.  A  small  section  covers, 
"What  Should  a  Warranty  Cost"  (1:4).  The  section  begins 
with  the  statement,  "It's  difficult  to  say  exactly  what  a 
warranty  should  cost"  (1:4).  It  then  suggests  the  analyst 
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should  ask  the  contractor  what  is  covered.  A  later  section 
describes  the  first  step  as  using  a  life-cycle  cost  model, 
and  then  proceeds  to  list  very  general  questions  to  answer, 
when  using  the  model ,  that  pertain  to  the  potential  costs 
and  benefits.  The  model  does  not  address  any  specific 
guidelines  for  performing  the  CBA.  The  contents  of  this 
warranty  guide  illustrates  the  need  for  stronger  guidance 
for  performing  CBAs. 

Comments  on  CBAs  -  Current  Status.  Brig  Gen  Lewis 
E.  Curtis,  III  published  an  article  in  the  Program  Manager 
Magazine,  "Time  for  a  Relook  at  Weapon  System  Warranties: 

The  Shotgun  Approach  Needs  to  Be  Replaced  With  More  Accurate 
Targeting"  (July-Aug  1990).  Gen  Curtis  began  by  saying, 
"...with  the  benefit  of  five  years,  it  is  time  to  revisit 
requirements  of  the  law  and  its  application"  (10:34). 

Weapon  system  warranties  do  not  make  sense  in  every 
procurement  situation  (10:34).  Specifically,  he  was 
referring  to  sole-source  and  cost-type  acquisitions.  Unless 
it  could  be  clearly  shown  to  be  beneficial,  he  said  it  makes 
good  business  sense  to  pursue  a  waiver  (10:37). 

During  1990,  Lt  Col  John  P.  Clarke  led  a  working  group 
to  define  an  implementable  warranty  program.  Although  the 
copy  of  the  report  that  was  available  was  not  dated,  it  is 
believed  to  have  been  published  in  early  1991.  Lt  Col 
Clarke  does  not  specifically  address  CBAs  in  his  report; 
however,  some  of  the  issues  covered  relate  to  a  cost 
analysis.  He  states  the  purpose  for  a  warranty:  "Clearly 
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the  government  is  too  large  to  be  protected  against 
catastrophic  financial  loss  by  any  contractor.  Only  the 
taxpayer  is  financially  more  capable  than  we  [the 
government]*'  (8:4).  He  questions  the  validity  of  saying  a 
warranty  is  not  cost  effective  because  an  item  did  not  break 
often  enough  during  the  period  warranted  to  recover  the 
cost.  In  contrast,  the  increased  reliability  may  indicate 
the  warranty  is  working.  Lt  Col  Clarke  concluded,  "In  sum, 
we  suggest  that  the  only  logical  purpose  for  our  obtaining  a 
warranty  is  to  improve  the  quality  of  the  product"  (8:5). 

A  personal  interview  was  conducted  with  Mr.  Stephen 
Dizek  of  The  Analytical  Sciences  Corporation  (TASC)  to 
discuss  the  CBA  methodology  used  for  the  SPW  warranty  .  In 
1987  Mr.  Dizek  performed  a  CBA  for  the  system;  and  this  was 
prior  to  availability  of  actual  test  data.  He  used  a  risk 
analysis  comparing  government  versus  contractor  potential 
risks.  Mr.  Dizek  stated  that  typically  a  combination  of 
several  accepted  CBA  methodologies  are  utilized.  The  choice 
depends  upon  availability  of  data  and  the  life  cycle  stage 
of  the  system.  He  said  the  analyst  should  have  an 
understanding  of  statistical  and  probabilistic  methods  to 
conduct  an  adequate  CBA  (13). 

On  5  Nov  1991,  a  telephone  interview  was  conducted  with 
Maj  Thomas  DiNino,  SAF/AQCS  .  Maj  DiNino  is  the  Air  Force 
warranty  focal  point  for  contracting  policies  and 
procedures.  The  discussion  regarded  Air  Force  approved 
warranty  waivers.  Maj  DiNino  stated  there  have  been  six  Air 
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Force  warranty  waivers  approved  to  date;  and  all  of  the 
waivers  were  for  systems  of  the  "one-time  use"  type  (12). 
Further  investigation  indicates  that  only  two  waivers  were 
approved  for  systems  in  full -rate  production.  The  other 
waivers  were  for  the  low-rate  initial  production,  and  is 
exempt  by  public  law  upon  request  by  the  agency. 

EPR  Warranty  Background 

The  EPR  warranty  is  described  at  this  point  because  it 
is  the  major  area  of  interest  in  the  review  of  warranty 
effectiveness  for  the  SFW.  A  system's  EPR  warranty 
typically  drives  the  major  portion  of  warranty  cost.  As  a 
weapon  system  moves  away  from  the  concept  exploration  stage, 
the  likelihood  of  influencing  design  to  affect  EPRs 
decreases  (11:4).  Once  a  system  is  in  the  engineering  and 
manufacturing  development  stage,  it  would  be  very  costly  for 
the  contractor  to  improve  essential  performance  if  it  does 
not  meet  the  requirements  and  the  customer's  expectations. 
Deficiencies  are  detected  through  operational  tests  and 
evaluations  (OT&E) .  Warranty  claims  can  be  made  for  test 
failures,  and  remedies  vary  with  different  contracts. 
However,  a  test  failure  does  not  always  prompt  the 
contractor  to  redesign  a  system  to  meet  the  EPR  requirement. 
The  contractor's  analysis  may  indicate  it  is  more  cost 
effective  to  pay  the  remedies  for  breaching  the  EPR 
warranty,  as  opposed  to  making  an  engineering  change  to  the 
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system  to  meet  the  EPR.  The  warranty  contract  should 
specifically  state  redesign  as  a  remedy  so  the  contractor 
has  an  obligation  to  perform  the  redesign  effort.  Zero 
failures  for  systems  should  be  a  target,  but  is  an 
unrealistic  expectation. 

Relationship  Between  EPR  and  Design  and 
Manufacturing  Warranties.  Product  improvement  efforts  are 
required  to  mature  reliability  and  maintainability 
characteristics  (15:61).  The  design  of  a  system  and  the 
manufacturing  processes  employed  have  a  direct  relationship 
with  the  EPR  warranty.  Reliability  is  designed  into  the 
system  during  the  development  stages  by  methods  such  as 
fault  tree  analysis,  simplification  of  design,  redundancy, 
derating,  and  using  more  reliable  components.  Manufacturing 
process  such  as  environmental  stress  screening  and 
statistical  process  control  can  improve  a  systems 
reliability  by  detecting  faulty  workmanship  and 
manufacturing  process.  The  manufacturing  process  can  then 
be  improved,  and  the  result  would  be  a  higher  level  of 
reliability.  These  relationships  are  discussed  to  point  out 
the  interrelationship  of  the  warranty  parts:  1)  defects  in 
material  and  workmanship,  2)  design  and  manufacturing,  and 
3)  essential  performance  requirements.  If  EPRs  are 
satisfactorily  met,  then  it  is  likely  that  the  other  parts 
of  the  warranty  would  be  positively  affected. 

EPR  Selection.  Operational  performance 
requirements  are  the  basis  for  the  EPR  warranty.  The 
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requirements  for  the  EPR  should  be  selected  based  on 
attributes  that  cannot  be  determined  with  certainty  prior  to 
or  during  acceptance  testing.  Verification  of  EPRs  occurs 
during  the  operations  phase  after  the  system  is  fielded. 

EPRs  should  be  measurable  and  verifiable  in  order  to  make 
valid  claims  and  to  perform  valid  CBAs.  Availability, 
mission  capability,  and  turn-around  time  are  examples  of 
potential  EPRs.  Clear  and  concise  requirements  and 
measurements  must  be  stated  in  the  warranty.  For  example,  a 
warranty  may  state  the  requirement  for  a  mean-time-between- 
maintenance  of  250  hours.  The  EPR  is  derived  from  the 
Operational  Requirements  Document  (ORD)  and  by  the  user  and 
is  the  basis  for  the  requirement  in  the  warranty  clause 
(11:4-5) . 

Essential  Performance  Parameters  (EPPs)  for 
Typical  Systems.  Essential  Performance  Parameters  (EPPs) 
for  a  typical  weapon  system  are  monitored  continuously  by 
analyzing  measurements  collected  during  normal  field 
operation  and  maintenance  of  the  weapon  system  (11:5).  An 
example  of  a  typical  system  is  an  aircraft  engine  that 
accumulates  flying  hours  on  a  steady  basis.  Maintenance  and 
repair  actions  are  tracked,  and  a  mean-time-between 
maintenance  can  be  calculated.  Also,  during  downtimes, 
estimates  of  the  spares  required  to  maintain  readiness  can 
be  calculated.  Aircraft,  vehicles,  avionics,  and  support 
equipment  can  all  be  monitored  in  a  similar  manner  because 
of  the  regularity  of  field  use. 
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EPPs  for  Dormant  Systems.  Warranty  administration 
for  dormant  weapon  systems,  such  as  the  Advanced  Cruise 
Missile  (ACM)  Propulsion  System,  indicates  that  EPPs  for 
dormant  systems  are  difficult  to  monitor  because  they  are 
not  operated  in  field  conditions  on  a  regular  basis  (24).  A 
dormant  system  usually  undergoes  a  preacceptance  test  prior 
to  delivery  to  the  government.  Following  delivery  the 
system  is  stored,  and  at  some  specified  point  in  time,  the 
depot  may  perform  service  maintenance  on  the  system. 

Service  maintenance  may  include  replacing  some  components  or 
performing  an  external  check  for  rust  and  humidity  damage; 
and  it  can  include  both  replacement  and  external  checks. 

The  system  may  be  tested  at  specified  intervals,  determined 
by  the  warranty  contract,  to  verify  the  EPPs  against  the 
EPRs .  Destructive  testing  is  the  only  test  option  for  some 
systems  while  test  sets  can  be  used  fcr  other  systems. 

Built-In  System  Redundancy 

A  discussion  of  the  SFW's  built-in  system  redundancy  is 
important  to  the  warranty  issue  because  it  may  have  an 
impact  on  the  reliability.  In  reliability  engineering, 
redundancy  can  be  defined  as  the  existence  of  more  than  one 
means  for  accomplishing  a  given  task  (3:197).  The  classic 
type  of  redundancy  is  employed  by  reducing  the  number  of 
single  point  failures  by  having  two  or  more  ways  for  the 
system  to  operate.  Active  redundancy  means  that  all  the 
redundant  components  are  continuously  energized  and  are 
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employed  in  the  performance  of  the  function  (3:199). 

Standby  redundancy  means  that  the  duplicate  elements  do  not 
perform  any  function  unless  the  primary  element  fails 
(3:199).  The  SFW  employs  redundancy  in  a  nonclassical 
sense.  Tactical  redundancy  results  from  the  system's 
ability  to  hit  the  same  target  more  than  one  time. 

The  quantifiable  and  non-quantif iable  costs  must  be 
considered  in  the  evaluation  of  the  cost-effectiveness  of 
redundancy.  The  non-quantif iabl e  costs  are  considered  to  be 
indirect,  but  should  be  evaluated  since  they  impact  total 
life-cycle  cost  of  the  system  (4:7-11).  The  quantifiable 
costs  would  be  incurred  with  the  increased  number  of  systems 
or  units  procured.  Additional  maintenance  could  occur  for 
reparable  systems.  The  non-quantif iable  costs  would  be 
incurred  in  the  additional  weight  and  complexity  of  a  system 
because  of  the  additional  units,  elements  or  systems 
involved.  However,  the  higher  investment  costs  must  be 
weighed  against  the  lower  operating  and  support  costs  and 
the  higher  overall  reliability  of  the  system.  The  high  up 
front  acquisition  life  cycle  costs  would  be  offset  by  the 
increased  mission  support  capabilities  of  the  system. 

Conclusion 

In  1984,  Congress  mandated  DOD  weapon  system  warranties 
through  the  Public  Law  98-525.  The  only  exceptions  to  the 
warranty  requirement  are  when  the  warranty  is  not  cost 
effective,  or  it  is  not  in  the  best  interest  of  national 
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defense.  Although  CBAs  are  required  as  an  internal  control 
procedure  to  show  taxpayer's  money  is  spent  wisely,  only 
limited  guidance  has  been  provided  to  assist  in  performing 
CBAs . 

Subsequent  reports  written  illustrate  significant 
improvements  are  needed  in  CBAs.  The  RAND  Corporation 
suggests  knowledge  of  a  connection  between  warranties  and 
improved  levels  of  reliability  and  maintainability  should  be 
present  prior  to  the  purchase  of  a  warranty  (15:65). 

Reviews  and  inspections  will  continue  as  warranties  become 
institutionalized,  and  it  is  evident  CBAs  will  continue  to 
be  criticized. 

The  GAO  report  issued  in  1989  said  the  government 
spends  hundreds  of  millions  of  dollars  on  warranties  every 
year  in  DOD  and  questions  whether  all  warranties  purchased 
were,  in  fact,  cost-effective.  Waivers  are  a  viable  option 
if  the  CBA  indicates  the  warranty  is  not  cost-effective. 

EPRs  and  system  redundancy  were  discussed  in  this 
chapter.  Both  of  these  issues  are  relevant  to  the  warranty 
effectiveness  question  for  the  SFW. 

The  next  chapter  describes  the  methodology  used  for  the 
research,  and  chapter  four  describes  the  findings.  Chapter 
five  presents  the  research  conclusions. 
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Ill .  Methodology 


Introduction 

This  chapter  describes  the  methodology  used  to  answer 
the  investigative  questions.  The  research  objective  was  to 
1)  analyze  the  unique  attributes  of  performance 
requirements  for  one-time-use  systems,  specifically  the 
Sensor  Fuzed  Weapon  (SFW)  System,  to  determine  how  they 
impact  the  cost-effectiveness  and  future  desirability  for  a 
warranty,  and  2)  review  SFW  and  other  dormant  system  CBAs 
to  determine  if  a  general  methodology  can  be  developed  to 
improve  warranty  CBAs.  Two  investigative  questions  were 
developed  to  answer  the  research  questions.  First,  what 
system  attributes  make  the  SFW  different  from  the  typical 
warranted  system,  and  how  do  these  attributes  impact 
warranty  effectiveness?  Second,  when  examining  the  cost- 
effectiveness  of  a  one-time-use  system,  what  factors  are 
similar  for  dormant  systems,  and  is  it  possible  to  develop  a 
general  CBA  methodology? 

Justification  for  Method  Selected 

The  requirement  for  a  weapon  system  warranty  is 
relatively  new,  and  a  limited  amount  of  guidance  for 
analysis  of  a  system  warranty  is  available.  The  SFW  is  used 
as  the  subject  for  analyzing  one-time-use  systems.  The  SFW 
is  currently  in  low-rate-initial  production  and  has  initial 
test  data  available.  Destructive  testing  is  used  for  this 
system;  the  system  is  no  longer  useable  following  the  test. 
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The  Combined  Effects  Munition  (CEM)  System  was  the 
predecessor  to  the  SFW,  from  the  warranty  point  of  view,  and 
served  as  a  baseline. 

Methodology 

Investigative  Question  One.  What  system  attributes 
make  the  system  different;  and  how  do  these  attributes 
affect  the  warranty?  This  question  prompts  research  into 
the  unique  system  attributes  that  affect  the  warranty 
requirement  for  a  one-time-use  system.  The  subsystems  for 
the  SFW  will  be  analyzed  by  investigating  the  past  history 
of  testing,  technological  developments,  stability,  and 
reliability.  The  life-cycle  stage  of  the  system  will 
indicate  if  a  warranty  provides  potential  improvements  in 
system  design  (11:4).  EPR  warranties  for  the  Advanced 
Cruise  Missile  (ACM),  ACM  engine.  Peacekeeper  Missile,  and 
Maverick  Missile  will  be  reviewed  for  additional  information 
on  dormant  systems  as  they  relate  to  the  warranty 
requirement.  The  investigation  may  reveal  similar 
attributes  between  the  dormant  systems  that  may  impact  the 
warranty  applicability  of  other  systems. 

Investigative  Question  Two.  Are  cost  analyses  for 
dormant  systems  similar;  and  is  it  possible  to  develop  a 
general  CBA  methodology?  To  address  the  second 
investigative  question,  the  CBA  for  the  SFW,  as  well  as 
other  above  mentioned  one-time-use  systems,  will  be 
examined.  Commonality  between  CBAs  may  indicate  that  a 
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general  methodology  can  be  applied.  The  categories  for 
costs  and  benefits  used  in  the  CBAs  will  be  compared  to 
determine  if  a  general  methodology  would  be  applicable  for 
dormant  system  CBAs. 

Support 

Support  for  the  research  is  provided  by  the  Sensor 
Fuzed  Weapon  and  Airfield  Attack  System  Program  Office, 
ASD/YB,  Eglin  APB,  Florida.  They  provided  funds  to  visit 
the  program  office  to  gather  information  and  data.  The 
Armament  Division  at  Ogden  AFB,  00-ALC/LIWB,  provided 
historical  data  on  the  CEM  System. 

Validation  and  Historical  Accuracy  of  Method 

Actual  data  available  for  dormant  weapon  systems  will 
be  used  to  perform  the  analyses.  Documentation  on  how  to 
conduct  a  warranty  analysis  on  a  one-time-use  system  does 
not  exist.  Therefore,  the  researcher  will  use  comparisons 
between  dormant  systems  on  which  to  base  conclusions. 

Summary 

This  chapter  described  the  methodology  to  answer  the 
investigative  questions.  Data  and  information  gathering  is 
ongoing,  and  the  proposed  methodology  is  viewed  as  the  most 
reasonable  based  on  current  information.  The  next  chapter 
will  describe  the  research  findings,  and  conclusions  will  be 
presented  in  chapter  five. 
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IV.  Findings 


Introduction 

Chapter  Three  provided  a  general  background  on 
warranties  and  discussed  current  issues.  The  focus  of  this 
research  is  on  the  essential  performance  requirements  (EPR) 
portion  of  the  warranty.  An  EPR  is  a  measurable, 
verifiable,  trackable,  and  enforceable  operating  or 
maintenance  capability  and  reliability  characteristic  of  a 
weapon  system  that  is  determined  to  be  necessary  for  the 
system  to  fulfill  the  military  requirement  for  which  the 
system  is  designed  (11:1).  The  intent  of  the  warranty 
requirement  must  be  examined  prior  to  presenting  the 
research  findings  since  chapter  three  gave  a  brief  warranty 
background  for  understanding.  Defining  the  intent  of  DOD 
warranties  provides  a  baseline  for  the  review  of  various  EPR 
warranties  in  this  chapter.  The  requirement  was  mandated  by 
Congress  in  1984.  The  general  intent  was  to  provide  a  means 
to  match  contractor's  promised  performance  specifications, 
during  the  bidding  process,  with  measured  performance  in  the 
field.  The  types  of  warranties  are  1)  an  assurance 
warranty  that  induces  the  manufacturer  to  produce  a  product 
or  system  that  performs  as  advertised  and  guarantees 
quality,  performance,  and  design  levels,  and  2)  A 
reliability  improvement  warranty  (RIW)  that  induces  the 
manufacturer  to  exceed  minimum  requirements  by  providing 
incentives.  The  systems  discussed  in  this  chapter  are  of 
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the  assurance  type.  To  differentiate  from  a  commercial 

warranty,  the  following  objectives  for  a  military  weapon 

system  warranty  are  provided. 

To  mature  the  weapon  system  to  a  point  where  there 
is  an  acceptable  probability  that  the  system  will 
deliver  the  full  measure  of  its  designated 
capability  whenever  called  upon  in  combat. 

(15:63) 

The  Air  Force  Regulation  for  warranty  acquisition  provides 

the  following  definition. 

Weapon  system  warranties  (WSW)  provide  the  Air 
Force  ways  to  motivate  contractors  to  design, 
produce,  and  deliver  quality  weapon  systems  as 
well  as  a  means  to  correct  defects  for  which  the 
contractor  is  responsible.  (11:1) 

The  findings  presented  in  this  chapter  will  reveal  how  a 

warranty  on  a  dormant  system  relates  to  the  warranty 

objectives,  and  if  a  general  cost-benefit  analysis 

methodology  can  be  developed  to  improve  CBAs.  These 

objectives  provide  a  basis  for  research  on  the  validity  of 

an  EPR  warranty  for  a  dormant  weapon  system.  A  dormant 

weapon  system  is  one  that  is  kept  in  storage  until  demand 

for  actual  usage  occurs.  Missiles,  bombs,  and  munitions 

typically  fall  into  this  category.  Actual  use  of  the  system 

results  in  its  destruction. 

Findings  for  Research  Question  One. 

Research  question  one's  objective  was  to  analyze  the 
unique  attributes  of  performance  requirements  of  one-time 
use  systems,  specifically  for  the  SFW,  and  determine  how  the 
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attributes  impact  future  desirability  for  a  warranty.  This 
section  discusses  the  research  findings. 

Application  of  EPRs  for  Specific  Dormant  Systems.  The 
EPR  warranties  for  the  Peacekeeper  Missile  Propulsion 
System,  Advanced  Cruise  Missile  (ACM),  ACM  Propulsion 
System,  and  Maverick  Missile  were  reviewed  to  see  if 
similarities  exist  between  dormant  systems.  The  following 
information  was  obtained  for  each  system:  1)  specific  EPRs 
warranted,  2)  type  of  tests  performed  to  determine  if  a 
breach  in  EPR  warranty  has  occurred,  and  3)  program  office 
comments  about  the  EPR  warranty.  Each  system  review  is 
presented  below,  and  a  total  assessment  follows  the  review. 

Peacekeeper  Missile  Propulsion  System.  The 
specific  EPRs  covered  under  the  warranty  were:  action  time, 
total  impulse,  instantaneous  thrust,  thrust  tailoff, 
ignition  delay,  thrust  vector  actuator  (TVA)  enable  time, 

TVA  action  time,  TVA  slew  rate,  TVA  duty  cycle,  stage  IV 
duty  cycle.  The  test  performed  was  operational  test  and 
evaluation  (OT&E).  OT&E  included  random  selection  and 
testing  of  deployed  missiles  from  those  on  alert  at 
operational  missile  wings.  OT&E  was  used  to  verify  the 
actual  performance  reliability  of  their  operational  missile 
fleet.  A  waiver  to  the  EPR  warranty  was  granted  in  1988 
based  on  non-cost  effectiveness  and  included  the  warranties 
for  Stage  IV  production  contract  and  beyond.  Comments 
regarding  the  warranty  were  obtained  from  the  waiver 
application . 
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There  is  little  or  no  opportunity  to  determine  if 
the  missile  will  successfully  perform.  The  OT&E 
program  would  not  be  useful  as  a  warranty 
enforcement  tool  because  the  data  obtained  are 
insufficient  to  pinpoint  the  specific  point  of 
failure  (if  any)  thus  allow  the  Government  to 
assign  fault  for  a  problem.  Instrumentation  on 
OT&E  tests  is  limited  to  that  on  the  operational 
missiles  so  SAC  can  obtain  a  valid  picture  of 
their  ICBM  weapon  system's  accuracy  and 
reliability.  No  additional  subcomponent 
performance  or  failure  analysis  instrumentation  is 
added  because  of  the  added  weight  for  such 
incorporation  and  the  associated  range  and  payload 
penalties.  (5:6) 

Advanced  Cruise  Missile  (ACM)  System.  The 

specific  EPR  warranted  is  stated  as  percentage  of  systems 

available.  Determination  of  a  breach  of  warranty  is  based 

on  assessed  distribution  of  the  missile's  reliability  during 

ground  tests.  The  specific  tests  are  loaded  launcher  pylon 

test  (LLPT)  and  system  integration  test  (SIT).  Comments 

obtained  are  as  follows: 

While  OT&E  flights  are  accomplished  on  the  ACM, 
they  are  too  infrequent  to  be  statistically 
relevant,  and  the  selection  of  test  missiles  is 
driven  by  an  operational  need  for  random 
selection.  Thus  the  program  office  cannot  be 
assured  that  a  sufficient  quantity  of  any  given 
contract  buy  will  be  flown  during  a  reasonable 
warranty  period.  In  conclusion,  the  ACM’s 
performance  on  ground  testing  remains  the  only 
measurable  essential  performance  parameter.  Since 
there  is  no  redundancy  in  the  missile's  design,  a 
quality  problem  will  impact  the  government.  (18) 

The  comments  above  state  that  the  OT&E  flights  were  too 

infrequent  to  be  statistically  relevant.  The  test  plan 

could  have  been  developed  to  be  statistically  relevant.  In 

addition,  random  selection  is  an  acceptable  sampling 

technique;  the  statement  above  implies  that  it  is  not  a 
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problem.  The  ACM  System  Program  Office  intends  to  submit  a 
total  warranty  waiver  application  for  the  latest  negotiated 
lot  buy  based  on  non-cost  effectiveness.  The  ACM  has  been 
in  limited  production  for  three  years. 

ACM  Propulsion  System.  The  specific  EPR  warranted 

was  defined  as  a  failure  rate.  The  types  of  tests  performed 

under  the  Warranty  Demonstration  Test  Program  were  the 

Product  Assurance  Test  (PAST),  Effectiveness  Verification 

and  Improvement  Program  (EVIP),  Operational  Test  Launch 

(OTL),  and  Run  as  Received  (RAR).  Comments  on  the  warranty 

were  obtained  from  the  waiver  application. 

Unlike  manned  or  reusable  drone  aircraft  engines, 
whose  perf^. nance  characteristics  may  be  evaluated 
at  regular  intervals,  cruise  missile  engines 
remain  dc.mant  in  the  field.  They  sit  dormant  for 
years  with  no  scrutiny  whatsoever  except  for 
periodic  electrical  tests  of  the  fuel  control. 

Any  other  actual  or  incipient  failures  would  go 
undetected  until  the  missile  is  actually  flown. 

It  would  be  difficult  to  recover  enough  hardware 
in  the  event  of  a  failure,  to  conclusively  prove 
what  element  of  the  engine  failed.  (24:3) 

The  last  statement  above  implies  that  hardware  must  be 

recovered  to  show  the  cause  of  engine  failure.  However,  the 

event  of  a  failure  should  be  sufficient  to  make  a  claim  of 

some  kind.  A  full  warranty  waiver  was  granted  for  the 

system  in  March  1991.  The  ACM  propulsion  system  was  in  the 

production  lot  III  buy  at  the  time  the  waiver  was  granted. 

Maverick  Missile  System.  The  specific  EPR 
warranty  states  that  ninety-six  percent  of  inspected 
missiles  shall  pass  an  inspection  cycle.  The  type  of  tests 
performed  are  visual  and  functional  testing  on  approved  test 
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sets.  Comments  were  obtained  from  an  interview  with  the 
warranty  manager. 

The  missile  has  been  in  production  for  more  than 
twenty  years  and  has  had  an  EPR  warranty  except 
for  the  low-rate  initial  production  lot.  The 
provisions  state  that  the  Air  Force  is  responsible 
for  the  first  four  percent  of  failures  (previously 
five  percent);  subsequent  failures  are  the 
responsibility  of  the  contractor.  The  missile 
program  has  never  had  a  breach  of  the  EPR 
warranty.  The  warranty  is  not  priced  separately, 
and  the  cost  is  included  in  the  total  missile 
cost.  The  program  office  feels  the  EPR  warranty 
has  provided  incentive  for  the  contractor  to 
improve  the  failure  rate,  as  evidenced  by  the 
recent  reduction  from  five  percent  to  four 
percent.  (19) 

Combined  Effects  Munition  System.  The  EPR 
warranty  states  that  the  function  rate  of  the  system  will 
not  degrade  below  a  specific  percentage.  The  function  rate 
is  defined  as  the  number  of  units  (bomblets)  that  function 
from  all  successful  dispenser  events  divided  by  the  total 
number  of  units  released  from  all  successful  dispenser 
events  minus  no-test  units.  Triennial  testing  of  30  systems 
per  production  lot  for  the  10-year  warranty  is  the  basis  for 
measuring  the  function  rate.  The  system  was  produced  by  two 
contractors  until  1990.  In  1990  the  system  came  under  a 
sole-source  contract.  To  date  no  warranty  claims  have  been 
made  against  the  EPR  warranty.  However,  a  claim  was  made 
against  a  defective  item. 

Assessment  of  System  Reviews.  The  systems  reviewed  are 
summarized  as  follows: 
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1.  The  Peacekeeper  Missile  System  Program  Office  was 
successful  in  obtaining  a  waiver  in  1988.  The  waiver  was 
for  the  EPR  warranty  only. 

2.  In  1991,  the  ACM  propulsion  system  was  granted  a 
full  warranty  waiver;  the  waiver  was  for  defects  in  material 
and  workmanship,  design  and  manufacturing,  as  well  as  the 
EPR  warranty. 

3.  The  ACM  Program  Office  is  preparing  to  submit  a 
full  waiver  request,  including  all  three  warranty  parts  as 
described  in  2.  above,  for  the  missile  system. 

4.  Although  the  Maverick  Missile  Program  Office  is 
satisfied  with  their  warranty,  can  they  be  positive  that  the 
warranty  is  cost  effective  if  the  warranty  is  not  separately 
priced?  The  costs  and  benefits  of  a  warranty  must  be 
compared  to  determine  cost-effectiveness.  Although  the 
warranty  is  not  separately  priced,  it  is  most  likely  built 
into  other  costs.  A  valid  cost-effectiveness  analysis  is 
not  possible  in  this  instance. 

5.  Two  views  of  warranty  effectiveness  for  the  CEM  can 
be  taken.  First,  the  EPR  warranty  has  been  effective  since 
the  system  is  performing  satisfactorily  with  no  failures. 

The  warranty  may  be  working  as  intended  since  the  contractor 
apparently  produced  a  system  that  has  not  failed.  Second,  a 
warranty  has  been  purchased,  but  the  government  is  not 
receiving  tangible  benefits  such  as  warranty  repairs; 
therefore,  the  warranty  is  not  cost-effective  because  the 
measurable  benefits  are  zero. 
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These  system  assessments  were  performed  to  assist  in 
the  review  of  the  Sensor  Fuzed  Weapon  System  (SFW)  EPR 
warranty . 

All  of  the  systems  discussed  in  the  previous  section 
are  one-time-use  systems  similar  to  the  SFW.  This  section 
focuses  on  the  Combined  Effects  Munition  (CEM)  since  it  is 
the  predecessor  to  the  SFW  from  the  warranty  point  of  view, 
and  therefore,  it  will  provide  the  best  history  for 
comparison  with  the  SFW.  A  brief  description  of  the  SFW  and 
CEM  will  be  given  in  the  next  paragraph  for  comparison 
purposes.  A  brief  history  of  both  systems  will  follow  the 
description . 

The  SFW  and  the  CEM  are  cluster  munitions  that  are 
unpowered  and  unguided  free-fall  devices,  stored  and 
transported  as  an  all-up-round  in  a  container.  Both  of  the 
systems  provide  a  conventional  force  multiplier  capable  of 
achieving  multiple  kills  per  pass.  The  difference  between 
the  systems  is  the  SFW  is  effective  against  armor,  while  the 
CEM  is  not. 

History  of  the  Combined  Effects  Munition  (CEM).  Review 
of  the  CEM  system  warranty  provides  a  baseline  for 
determining  applicability  of  the  EPR  warranty  to  the  SFW. 

A  report  on  CEM  Warranty  Evaluation  was  published  in 
November  of  1990  by  ARINC  Research  Corporation.  Production 
of  the  system  began  in  1984  with  Aerojet  Ordnance  as  the 
prime  contractor  and  Honeywell  as  a  subcontractor  for  TMD 
production.  Dual  source  production  began  in  1985.  In  1991 
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the  program  went  sole-source  to  Honeywell  because  of  the 
high  cost  of  high  cost  of  maintaining  dual  source  production 
and  the  changing  political  scene  in  Europe  (2:1-4).  The  CEM 
program  office  developed  the  warranty  during  the  latter 
stages  of  engineering  and  manufacturing  development. 

The  basic  format  was  that  of  a  storage  verification 
warranty  with  penalties  for  the  contractors  (2:1-5).  The 
current  warranty  clause  provides  for  liquidated  damages 
(monetary  payments)  to  the  Government  should  the  product 
fail  to  meet  a  reliability  requirement  of  87%,  at  the  system 
level,  based  upon  triennial  flight  testing.  It  also 
provides  a  vehicle  for  the  Government  to  require  "no-cost" 
engineering  changes  for  future  production  if  testing 
indicates  that  the  reliability  has  not  reached  or  is  falling 
below  the  prescribed  levels  (2:1-6).  Cost  of  processing  and 
evaluating  ECPs  will  be  absorbed  by  the  contractor.  The 
costs  associated  with  implementing  and  performing  the  ECPs 
are  covered  in  the  contractor's  total  liability  limits 
(2:B-8).  The  warranty  provides  the  contractors  protection 
by  limiting  their  liability  to  approximately  eight  to  ten 
percent  of  the  total  contract  value.  Financial  catastrophe 
to  the  contractor's  business  cannot  occur  under  this 
warranty  contract. 

As  part  of  the  review  for  the  report,  several  questions 
were  asked  of  contractor  and  Government  procurement 
personnel  relating  to  whether  or  not  the  warranty  motivated 
the  contractors  to  provide  a  better  product.  Government 
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personnel  were  generally  non-committal,  although  several 
indicated  that  they  sensed  little  motivation  as  a  result  of 
the  warranty  (2:2-3).  The  contractors,  on  the  other  hand, 
stated  that  the  warranty  was  a  significant  factor  in  the  way 
that  they  approached  the  CEM  program  and  their  obligation  to 
provide  a  quality  product  to  their  customer.  Aerojet 
Ordnance  lobbied  for  the  inclusion  of  a  warranty  as  part  of 
their  marketing  strategy  (2:2-3).  Once  the  warranty  was  in 
effect,  ARINC's  review  of  the  procurement  contracting 
officer's  (PCOs)  warranty  files  indicated  only  two  letters 
in  the  entire  file  relating  to  directed  warranty 
modifications  to  the  CEM.  Although  one  contractor  indicated 
that  they  had  performed  a  significant  amount  of  analysis  of 
component  designs  prior  to  initially  signing  up  to  the 
warranty,  ARINC  saw  little  evidence  that  they  had  used  the 
results  of  their  analysis  to  alter  or  improve  component 
design  (2:2-3).  However,  if  the  design  was  already 
acceptable,  the  contractor  may  not  have  the  need  to  improve 
design.  In  this  case,  the  warranty  would  provide  protection 
against  catastrophic  failure,  similar  to  insurance  coverage. 

Sequential  Bayesian  testing  techniques  were  used  to 
reduce  test  costs.  This  method  enables  up  to  two-thirds  of 
the  flight  tests  to  be  avoided  because  a  decision  to  accept 
the  lot  can  be  made  before  the  maximum  amount  of  destructive 
testing  is  performed.  An  accept  or  reject  decision  can  be 
made  based  upon  the  number  of  test  failures  per  a  specific 
number  of  tests  performed.  A  numerical  estimate  of  true 
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reliability  would  require  a  larger  sample  size  and  would 
yield  higher  statistical  accuracy  (2:2-4).  According  to  the 
test  plan,  using  an  example  of  ten  tests  performed,  the  lot 
is  accepted  if  zero  failures  occur  in  ten  tests.  If  three 
failures  occur  in  ten  tests,  the  lot  is  rejected.  The 
maximum  number  of  tests  is  30,  if  results  continue  to  fall 
in  the  region  between  accept  and  reject.  Refer  to  Figure  1 
for  an  illustration  of  the  sampling  plan. 


Figure  1.  CEM  Sequential  Sampling  Plan 


ARINC  performed  a  cost-benefit  evaluation  on  the  CEM 
and  estimated  system  life  cycle  warranty  costs  of  $15 
million  and  potential  warranty  benefits  of  $22  million. 
"The  evaluation  showed  the  savings  accrued  through  lower 
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testing  costs  rather  than  any  clear  and  quantifiable 
improvements  brought  about  by  warranty  incentives"  (2:2-7). 
To  date,  no  warranty  claims  have  occurred  under  the  EPR 
warranty.  Since  no  claims  have  occurred,  the  CEM  appears  to 
be  an  acceptable  operational  system.  However,  it  cannot  be 
shown  that  the  warranty  had  an  effect  on  the  system 
reliability. 

History  of  the  Sensor  Fuzed  Weapon  (SFW)  System. 

Primary  differences  between  the  systems  are  the  SFW 
dispenser  skin  peels  and  the  nose  and  tail  remain;  the  CEM 
dispenser  spins  and  ejects  the  bomblets  while  the  SFW  uses 
the  tactical  munitions  dispenser  (TMD)  fuze  with  no-spin 
settings.  The  SFW  program  milestones  are  as  follows: 

1.  Initiated  in  June  1984  as  a  risk  reduction  phase 
with  an  option  for  full  scale  development  (FSD). 

2.  In  November  of  1985,  the  FSD  option  was  exercised. 

3.  The  critical  design  review  was  conducted  in  1989. 

4.  The  live  fire  test  and  evaluation  was  performed  in 
1990. 

5.  A  production  transition  program  was  initiated  in 
July  of  1990. 

6.  The  developmental  test  and  evaluation  (DT&E)  was 
successfully  conducted  between  December  1988  and 
April  1991. 

7.  Following  the  DT&E,  initial  operational  test  and 
evaluation  (lOT&E)  was  successfully  completed  in 
December  1991. 
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8.  The  full-scale  production  is  estimated  to  begin  in 
1995. 

The  Analytical  Sciences  Corporation  performed  a 

warranty  analysis  on  the  SFW  in  1987.  Equipment 

characteristics  were  reviewed,  and  similarities  with  CEM 

were  identified  as  follows: 

As  a  "one  shot"  device,  it  has  an  operating  time 
so  short  that  reliability  is  not  affected  by  wear- 
out  or  time  during  operation.  Reliability  depends 
on  the  probability  that  parts  will  function 
properly  at  the  moment  required.  (14:3-2) 

As  in  CEM,  SFWs  are  non-operating,  non-reusabl e , 
and  non-restorable  systems.  Consequently, 
application  redundancy,  pipeline  spares  and 
maintainability  as  influences  on  availability  are 
generally  irrelevant.  (14:3-2) 

Transportation  costs  are  the  major  component  of 
SFW  operations  and  maintenance  costs,  and  are 
relatively  fixed.  O&M  represents  only  one-percent 
of  the  total  life  cycle  costs.  Thus,  the 
opportunity  to  focus  on  O&M  costs  with  a  warranty 
is  minimal.  (14:3-3) 

Automated  and/or  nondestructive  test  techniques 
that  monitor  the  "health"  of  the  system  will  not 
be  used  on  SFWs  once  they  are  stored  at 
operational  locations.  Such  information  on  SFW 
must  be  gathered  primarily  through  discrete, 
destructive  testing.  (14:3-3) 

SFW  will  be  stored  for  long  (up  to  10  years,  or 
longer)  periods  of  time,  and  then  be  expected  to 
function  whenever  required.  The  "all  up  round"  -- 
build  it,  store  it,  don't  disturb  it,  use  it  when 
required  --  is  specifically  applicable  to  SFW  as 
it  was  for  CEM.  (14:3-3) 

TASC  performed  an  analysis  to  determine  the  overall 
effect  of  a  warranty  on  the  SFW  program.  A  comparison  was 
made  between  government  economic  risk  without  a  warranty  and 
the  government  economic  risk  with  a  warranty.  The  analysis 
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concluded  that  there  is  a  significant  benefit  in  having  a 
warranty  for  the  correct  price,  over  not  having  one.  The 
conclusion  was  based  on  the  assumption  that  the  warranty 
would  motivate  the  contractor  to  build  SFHs  that  degrade  at 
0.01  per  year,  versus  0.021  per  year  without  a  warranty 
(14:4-5) . 

Specific  Findings  for  the  SFW.  The  methodology  used  to 
measure  EPR  effectiveness  for  the  SFW  is  a  test  of 
hypothesis  using  the  t-statistic.  Six  systems  from  each 
full-rate  production  lot  buy  will  be  tested  at  the  four  year 
and  eight  year  points.  The  12  systems  are  tested  based  on 
the  assumption  that  the  characteristics  of  the  sample  will 
resemble  population  characteristics. 

The  sample  is  limited  because  testing  a  large  sample  is 
cost  prohibitive.  The  unit  cost  could  be  as  high  as 
$249,000.00  each,  and  testing  12  results  in  destruction  of 
systems  costing  $2.98M  each  lot,  since  destructive  testing 
is  utilized.  The  attribute  being  measured  is  degradation  of 
the  system  over  time. 

Table  1,  on  the  following  page,  contains  the  total 
program  quantities  by  year.  Low-rate  production  occurs 
between  1992  and  1995,  and  521  systems  will  be  acquired. 

The  system  will  then  be  in  full-rate  production  between  1996 
and  2004,  and  9,479  systems  will  be  acquired  during  this 
period.  The  total  quantity  to  be  produced  is  10,000 
systems . 
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TABLE  1 


SFW  PROGRAM  QUANTITIES 
BY  YEAR 


FISCAL 

YEAR 

LOTS 

NUMBER  OF 

1992 

LRIP  1 

98 

1993 

LRIP  2 

23 

1994 

LRIP  3 

175 

1995 

LRIP  4 

225 

1996 

FRP  1 

480 

1997 

FRP  2 

870 

1998 

FRP  3 

1156 

1999 

FRP  4 

1236 

2000 

FRP  5 

1330 

2001 

FRP  6 

1366 

2002 

FRP  7 

1439 

2003 

FRP  8 

1447 

2004 

FRP  9 

155 

LRIP: 

1 ow-rate 

initial  production 

FRP: 

full-rate 

production 

The  current  test  plan  for  the  SFW  calls  for  the  limited 
testing  of  a  maximum  of  six  systems  at  the  four  and  eight 
year  points  following  delivery  of  the  system.  According  to 
the  program  manager,  the  high  cost  of  destructive  testing 
prohibited  the  use  of  sequential  sampling.  For  comparison 
purposes,  the  MX  ballistic  missile  costs  for  each  follow-on 
operational  test  and  evaluation  (FOT&E)  is  estimated  at  $85 
million  per  launch.  The  ground  tests  include  some 
destructive  and  some  nondestructive.  The  program  office  was 
forced  to  reduce  the  number  of  flight  tests  from  seven  to 
three  and  rely  on  ground  tests  and  analysis  for  estimates  of 
degradation  (6:42).  Although  the  test  costs  for  the  SFW 
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are  much  smaller,  limited  funding  can  significantly  reduce 
the  number  of  tests  performed. 


The  weakness  in  using  the  t-statistic  for  the  SFW's 
measurement  of  degradation  is  that  a  small  sample  provides 
the  contractor  more  leeway  because  of  the  large  acceptance 
region  for  the  test.  A  larger  sample  size  and  tighter 
acceptance  region  could  have  the  potential  to  make  the 
contractor  more  responsible.  In  addition,  the  measurement 
of  test  results  occurs  with  each  of  the  two  sets  of  six 
tests  for  each  lot.  The  result  of  this  method  of  testing  is 
a  "point-in-time"  measurement  over  all  of  the  lots  and  not  a 
cumulative  measurement  that  shows  improvements  over  time. 
This  method  of  measurement  could  potentially  result  in  a 
more  costly  warranty  because  the  contractor  is  liable  for 
meeting  requirements  based  on  each  individual  test,  and  the 
result  is  higher  risk  for  the  contractor.  On  the  other 
hand,  a  cumulative  measurement  point,  or  points,  would 
translate  to  a  lesser  number  of  times  for  potential  invoking 
of  remedies. 

The  decision  to  limit  testing  to  12  systems  and  use  the 
t-statistic  was  based  on  the  cost  of  destructive  testing. 

The  resulting  reduction  in  potential  contractor  liability 
would  be  considered  in  an  analysis.  Tradeoffs  looking  at 
the  increased  confidence  in  the  test  of  hypothesis  verses 
the  resulting  increased  cost  of  testing  would  also  be 
considered  in  an  analysis. 
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Another  factor  to  be  considered  is  the  niunber  of 
systems  tested  relative  to  the  production  time  of  the 
system.  Potential  exists  for  the  contractor  to  close  down 
the  production  line  following  the  last  production  lot  in 
2006.  Only  forty-eight  systems  from  production  lots  1996- 
2000  would  have  been  tested,  leaving  72  from  the  remaining 
production  lots  to  be  tested  after  the  production  line  is 
closed.  Modifications  to  the  system  could  occur  prior  to 
closing  of  the  production  line.  However,  after  the 
production  line  is  closed,  only  monetary  remedies  could  be 
collected  from  the  contractor.  The  drawback  to  receiving 
monetary  remedies  is  that  the  U.S.  Treasury,  and  not  the 
Air  Force  or  program  office,  is  the  recipient  of  the  funds. 
Funds  to  correct  any  deficiencies  would  have  to  come  from 
some  other  source  since  the  contractor's  remedy  payment  goes 
to  the  U.S.  Treasury.  Refer  to  Appendix  A  for  details  of 
the  test  plan. 

The  SFW  System  has  some  redundancy  characteristics. 

At  the  system  level,  a  typical  load  of  four  SFW's  (CBU-97B) 
are  carried  on  an  aircraft.  Each  SFW  contains  ten  TMD's 
(BLU-108/B).  Each  of  the  ten  TMDs  contains  four 
projectiles.  The  total  nximber  of  projectiles  per  SFW  is  40, 
so  the  typical  load  of  four  SFW's  would  contain  160 
projectiles.  In  addition  to  the  redundant  components, 
tactical  redundancy  also  exists  because  of  the  possibility 
for  a  target  to  be  detected  and  hit  more  than  one  time. 
Redundancy  was  designed  into  the  system,  but  it  was  not 
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specified  as  a  requirement  to  meet  the  minimum  performance 
level . 

The  drawbacks  of  additional  weight  and  complexity  do 
not  affect  this  system.  The  SFW  is  a  1000  pound  class 
munition  of  common  design,  and  the  user  requirements  of  four 
SFWs  per  F-16  aircraft  to  eight  SPHs  per  F-111  aircraft  are 
within  the  weight  allowances.  Also,  the  SFW  is  a 
nonreparable  system  and,  therefore,  additional  maintenance 
is  not  a  factor.  The  warranty  test  at  years  four  and  eight 
for  each  lot  consists  of  destructive  testing  of  six  systems. 
Maintenance  actions  or  remove  and  replace  actions  do  not 
occur.  This  system  benefits  from  the  additional  reliability 
without  suffering  any  of  the  potential  drawbacks  of 
redundancy. 

The  formula,  proposed  by  the  system  program  office  for 
system  reliability  that  considers  redundancy,  appears  in 
Appendix  B.  The  subsystem  reliability  inputs  to  the  formula 
were  based  on  an  average  from  repeated  simulations.  The 
simulations  utilized  a  scenario  that  was  accepted  by  the 
user  as  an  expected  mission  scenario.  The  using  command 
made  the  decision  to  use  a  formula  for  reliability  for  a 
series  system.  A  series  system  formula  is  simply  the 
reliability  of  each  subsystem  multiplied  together  to  equal 
the  system  reliability.  The  formula  in  Appendix  B  is  more 
appropriate  because  it  models  the  tactical  redundancy 
characteristics,  while  the  series  formula  does  not. 
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The  life-cycle  stage  of  the  SPW  system  was  considered 
important  in  this  research.  Systems  in  the  early 
development  stage,  when  the  system  is  not  proven,  can 
benefit  from  the  EPR  warranty.  The  warranty  provides  the 
contractor  with  a  goal  for  meeting  the  stated  requirements 
for  performance.  Design  and  manufacturing  processes  can 
both  be  influenced  early  in  a  program.  As  the  system 
matures  and  design  stabilizes,  only  manufacturing  processes 
can  be  influenced.  A  system  that  has  been  fielded  for  many 
years  will  not  benefit  as  greatly  from  the  warranty,  as  will 
an  unproven  system.  Several  of  the  components  of  the  SFW 
system  have  had  many  years  of  testing,  as  well  as  Desert 
Storm  usage  behind  them.  An  extensive  full-scale 
development  effort  for  the  SFW  began  in  1985.  Some  of  the 
common  components  between  the  CEM  and  the  SFW  are  the  FZU- 
39/B  proximity  sensor,  TMD  internal  fuze,  TMD  dispenser,  and 
the  CNU  411  A/E  weapons  container.  The  TMD  dispenser  is  a 
component  that  is  not  redundant  and  could  cause  a 
catastrophic  failure  if  it  did  not  operate  properly. 

However,  it  has  been  produced  successfully  for  over  10 
years . 

Considering  all  the  field  experience  for  the  CEM,  the 
EPR  has  been  successfully  met  to  date.  Thirty  DT&E  and 
twenty-five  lOT&E  tests  have  been  performed  for  the  SFW 
System.  These  tests  confirm  the  basic  design  of  the  system. 
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Findings  for  Research  Question  Two 


Research  question  two's  objective  was  to  review  the 
SFW's  and  other  dormant  system's  CBAs  to  determine  if  a 
general  methodology  could  be  developed  to  improve  CBAs. 

This  section  provides  the  findings  for  research  question 
two.  The  first  section  provides  a  general  overview  of  the 
methodologies  used  for  the  CBAs  for  Peacekeeper  Missile, 

ACM,  ACM  Propulsion  System,  Maverick  Missile,  CEM,  and  the 
SFW.  Next,  the  effects  of  sole-source  contracts  are 
discussed. 

System  CBAs  Described.  The  Peacekeeper  Missile  CBA  is 
described  as  follows:  For  warranties,  the  benefit  derived 
is  the  cost  avoidance  to  the  Government  that  results  from 
having  the  contractor  perform  repair  and  replacement  actions 
or  correct  problems  at  his  own  expense,  plus  the  cost 
avoidance  from  not  having  to  maintain  an  organic  repair  shop 
and  train  and  pay  repair  technicians.  The  cost  of  warranty 
is  that  of  buying  coverage  from  the  contractor  plus  the  cost 
of  a  system  to  document,  track,  and  report  claims  against 
the  warranty  (5:17). 

Factors  included  in  the  analysis  were  cost  of 
acquisition,  cost  of  preparing  or  setting  up  each  item 
(aircraft,  missile,  ship,  truck,  computer,  etc.)  for 
operation,  cost  of  routine  preventative  maintenance,  cost  of 
failure  repair,  cost  of  spare  parts,  and  the  cost  of  having 
or  contracting  a  repair  shop,  including  labor  (5:17). 
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Comparison  of  life  cycle  cost  with  and  without  warranty  was 
performed. 

The  Advanced  Cruise  Missile  (ACM)  CBA  is  described  as 
follows:  The  ACM  System  Program  Office  (SPO)  maintains 
reliability  assessments  of  ground  test  reliability  for  Level 
1  tests  in  the  form  of  a  cumulative  distribution  function. 
The  probability  of  the  missile  failing  a  Level  1  test  by  a 
given  percentage  is  extracted  from  a  distribution  function. 
Next,  each  percentage  is  applied  to  a  Cumulative 
Availability  Rate  (CAR)  formula  to  determine  a  breech  or 
compliance  with  CAR  provisions.  Given  a  breech,  the  cost  of 
a  warranty  engineering  change  proposal  (ECP)  (determined 
from  program  historical  data)  was  added  to  the  calculated 
risk.  Then  a  percentage  of  the  total  missile  failure  rate 
was  assvuned  to  be  corrected  by  the  ECP  (in  the  year  the  CAR 
was  calculated)  and  a  new  failure  rate  is  established.  This 
compiled  failure  rate  was  reapplied  to  the  next  CAR  period 
and  the  process  was  repeated  until  either  the  CAR  exceeds 
the  requirement  or  the  warranty  period  is  expired. 
Sensitivity  analyses  were  performed,  varying  the  failure 
rate  from  10  to  40  percent.  The  cost  of  money  was  included 
in  the  CBA.  The  last  CBA  performed  indicated  that  one  of 
the  contractor's  warranties  (dual  sourced)  was  not  cost- 
effective,  and  the  program  office  is  currently  seeking  a 
waiver . 

The  ACM  Propulsion  System  CBA  is  described  as  follows: 
The  CBA  for  the  ACM  propulsion  system  was  based  on 
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historical  data  for  that  system,  as  well  as  the  predecessor 
system.  The  data  consisted  of  the  nximber  and  types  of 
failures  that  occurred,  and  the  cost  of  repair  for  those 
failures.  In  addition,  an  engineering  projection  for 
reliability  and  projected  overall  failure  rates  were  used. 

A  sensitivity  analysis  was  performed  comparing  expected 
costs  and  benefits  against  a  worst  case  scenario  in  which 
the  benefits  would  be  lower  than  expected.  The  sensitivity 
analysis  was  important  because  the  SPO  was  requesting  a 
waiver  to  the  warranty.  Non-quantifiable  factors  such  as 
missile  deliveries  falling  behind  schedule  were  included  in 
the  narrative.  The  lagging  delivery  of  missiles  caused  the 
loss  of  warranty  coverage  time  for  the  engines  .  The 
narrative  also  included  the  judgement  that  the  probability 
of  exceeding  the  liability  ceiling  was  very  low,  thus, 
representing  a  low  risk  to  the  contractor.  A  comparison  of 
life  cycle  costs  with  and  without  the  warranty  was  made. 

The  last  CBA  performed  indicated  the  warranty  was  not  cost- 
effective,  and  a  waiver  to  the  total  warranty  was  obtained 
in  March  of  1991. 

The  Maverick  Missile  System  CBA  is  described  as 
follows:  The  Maverick  Cost-of-Ownership  Model  (MAVCOM) 
calculates  all  support  life  cycle  costs  (cost  of  ownership) 
associated  with  maintaining  the  inventory  of  the  Maverick 
missile  throughout  its  operational  life.  The  model  assesses 
the  cost  differences  associated  with  incorporating  an 
engineering  change  into  production  or  with  changing  an 
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operational  or  support  parameter.  A  baseline  failure  rate 
is  used  because  the  government  is  responsible  for  failures 
up  to  four  percent.  A  five  percent  failure  rate  is  then 
calculated  to  obtain  the  costs  to  repair  warranted  failures. 
The  predicted  repair  actions  are  the  delta  between  the 
failure  rates  at  four  and  five  percent. 

The  SPO  has  never  e'^.covmtered  a  breach  of  the  EPR 
warranty,  and  the  warranty  is  not  separately  priced.  It  is 
questionable  that  a  warranty  can  be  identified  as  cost- 
effective  if  it  is  not  separately  priced.  However,  the  SPO 
feels  that  it  has  provided  incentive  for  the  contractor  to 
reduce  the  failure  rate  from  five  percent  to  four  percent. 
The  missile  has  been  in  existence  for  over  twenty  years. 

Combined  Effects  Munition  (CEM)  CBA  is  described  as 
follows:  The  CBA  for  the  CEM  compares  the  government  cost 

with  and  without  a  warranty  for  the  following  factors: 
warranty  price,  warranty  administration,  test 
administration,  conduct  of  testing,  and  ECPs.  The  analysis 
indicated  that  government  costs  were  higher  without  a 
warranty  due  to  the  higher  cost  of  test  administration  and 
higher  cost  of  conducting  tests  Therefore,  the  purchase  of 
a  warranty  was  determined  to  be  cost-effective.  To  date,  no 
EPR  warranty  claims  have  been  made. 

Sensor  Fuzed  Weapon  (SFW)  System  CBA  is  described  as 
follows:  The  CBA  for  the  SFW  identifies  differences  in 

degradation  per  year  and  reliability  over  the  warranty 
period  with  and  without  a  warranty.  The  expected  value  of 
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"good"  assets  was  computed  based  on  the  differences 
previously  mentioned.  Many  sensitivities  were  performed  by 
changing  the  degradation  rate,  reliability,  and  contractor 
liability  cap.  The  cost  of  money  was  considered  by 
discounting  using  ten  percent  per  year.  Expected  value  of 
"good"  assets  at  ten  years,  with  and  without  a  warranty,  was 
computed.  Net  warranty  CBA  was  the  difference  between  total 
EPR  warranty  cost  and  total  expected  EPR  warranty  pay  back 
plus  total  expected  value  of  "extra  good"  assets.  The  SFW 
is  not  a  reparable  system,  and  the  analyst  determined  to  use 
the  value  of  "good"  assets  with  and  without  a  warranty  to 
perform  the  CBA.  A  narrative  that  describes  many  non- 
quantifiable  costs  or  factors  that  affect  the  warranty  is 
included  with  the  CBA.  The  CBA  indicated  that  the  warranty 
is  not  cost-effective. 

Summary  of  the  CBAs.  Appendix  C  compares  the  cost  and 
benefit  categories  used  for  each  of  the  warranties 
discussed.  The  typical  method  of  computing  warranty 
benefits  is  to  project  repair  costs  for  the  period 
warranted.  The  benefits  are  then  compared  to  the  cost  of 
the  warranty,  or  potential  cost  of  projected  repair,  to 
arrive  at  a  benefit  to  cost  ratio  or  delta.  The  SFW  is 
distinguished  from  the  other  systems  because  it  is 
nonreparable.  The  difficulty  for  a  dormant,  nonreparable 
system  is  determining  what  to  measure  to  represent  benefits. 
The  method  of  using  estimated  "good"  assets  is  a  reasonable 
measure . 
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Sole-Source  Versus  Competitive  Contracts.  A  sole- 
source  contract  poses  difficulties  relative  to  the  warranty 
issue.  An  increasing  number  of  firms  use  warranties  as  an 
offensive  marketing  weapon  that  is  an  integral  part  of  their 
overall  strategy.  The  objective  is  to  differentiate  the 
product  in  a  competitive  environment  (17:69).  Understanding 
the  driving,  hostile  forces  of  a  competitive  environment, 
management  recognizes  the  necessity  of  acquiring  new  and 
repeat  customers  (20:44). 

Loss  of  government  leverage  occurs  in  a  sole-source 

environment.  The  SFW  system  program  was  down-selected  to  a 

single-source.  The  negotiation  of  an  effective  warranty 

becomes  difficult  in  this  scenario. 

Sole-source  acquisitions  pose  a  variety  of  unique 
challenges.  Without  pressures  of  the  marketplace, 
difficulties  of  assuring  a  fair  and  reasonable 
price  are  magnified.  In  the  warranty  area,  too, 
dealing  with  a  single-source  requires  special 
considerations.  The  government  leverage  in 
assuring  a  reasonable  assessment  of  risk  by  the 
contractor  is  significantly  reduced.  Without 
competition,  the  contractor  will  take  an  extremely 
conservative  approach,  which  translates  into  high 
warranty  costs  or  low-performance  parameters,  or 
both.  Flexibility  given  by  the  law  to  negotiate 
is  a  double-edged  sword  which,  in  a  sole-source 
environment,  can  lead  to  extended  negotiations 
where  the  contractor  can  wear  down  the  government 
until  it  gets  a  meaningless,  watered-down  warranty 
simply  to  satisfy  requirements  of  the  law. 

(10:35) 

Sole-source  contractors  do  not  have  the  motivation  to 
provide  a  good  warranty  to  differentiate  themselves  from 
competitors.  The  waiver  authority  under  the  law  should 
provide  relief  in  the  situation  that  a  warranty  from  a  sole- 
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source  contractor  is  not  cost-effective.  The  GAO  criticized 


the  Air  Force  for  conducting  extended  negotiations  rather 
than  requesting  a  waiver  (10:35). 

A  contractor  has  to  accept  high  risk  on  a  conceptual 
system  utilizing  leading  edge  technology.  The  complexities 
of  a  new  system  demand  greater  risk  coverage,  and  therefore, 
higher  warranty  costs.  However,  as  discussed  in  the  system 
history,  the  SFW  is  a  proven  system  for  most  of  the 
components,  including  the  ones  that  could  cause  a 
catastrophic  failure.  The  contractor  for  the  system, 
nevertheless,  is  risk  averse  as  evidenced  by  the  anticipated 
high  cost  of  warranty  compared  to  anticipated  benefits.  The 
non-competitive  aspect  of  the  system  can  be  a  major  factor 
in  the  contractor  not  accepting  risk. 

Summary 

This  chapter  presented  the  findings  for: 

1)  application  of  essential  performance  requirements  (EPR) 
warranty  relative  to  the  SFW,  and  2)  review  of  cost-benefit 
analyses  to  determine  if  a  general  methodology  could  be 
developed  were  addressed.  The  next  chapter  presents 
conclusions  and  recommendations  for  the  research  questions. 
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V. 


Introduction 

The  previous  chapter  covered  detailed  findings  for  this 
research.  This  chapter  sununarizes  the  conclusions  and 
provides  recommendations. 

Conclusions 

The  first  objective  of  this  research  was  to  analyze  the 
unique  attributes  of  the  Sensor  Fuzed  Weapon  (SFW)  System's 
performance  requirements  to  determine  how  they  impact  the 
cost-effectiveness  and  future  desirability  for  a  warranty. 
The  SFW  program  office  is  considering  waiving  the  EPR 
warranty  requirement  based  on  the  most  recent  CBA  that 
indicated  the  warranty  was  not  cost-effective.  They  are 
justified  in  questioning  the  purchase  of  an  EPR  warranty. 

The  conclusions  based  on  the  areas  reviewed  are  presented 
below. 

Review  of  similar  dormant  weapon  systems  indicated  a 
unique  attribute  was  the  method  of  testing  (destructive 
testing)  for  measurement  of  essential  performance 
requirements.  Destructive  testing  was  required  and  resulted 
in  a  limited  number  of  tests  being  performed  to  detect  if  a 
system's  performance  requirements  were  satisfactorily  met. 
Destroying  systems  for  test  purposes  made  it  cost 
prohibitive  to  have  a  large  test  sample.  Two  of  the  systems 
reviewed  had  warranty  waivers  granted  based  on  the  CBA 
indicating  the  warranties  were  not  cost-effective.  The 
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rationale  included  the  dormant  nature  of  the  system,  as  well 
as  limited  testing  opportunities  due  to  random  selection  and 
destructive  testing.  The  SFW's  testing  is  limited,  and  if 
the  production  line  closes  in  2006  as  projected,  less  than 
half  of  the  total  population  of  systems  would  have  been 
subjected  to  testing  by  the  end  of  the  production  warranty. 
After  the  production  line  closes,  the  only  remedy  available 
for  a  breech  in  warranty  would  be  monetary  and  would  not 
result  in  a  system  improvement.  The  monetary  remedy  would 
then  most  likely  go  to  the  U.S.  Treasury  and  would  not 
benefit  the  program  or  the  Air  Force.  Also,  a  breach  in  the 
warranty  contract  is  based  on  the  method  of  measuring 
essential  performance  parameters  at  ”points-in-time” .  This 
method  may  drive  up  warranty  cost  because  of  higher  risk  to 
the  contractor.  An  alterative  would  be  to  measure 
performance  on  a  cumulative  basis. 

The  CEM,  SFW's  predecessor  system  in  the  warranty  point 
of  view,  has  had  no  claims  against  the  EPR  warranty. 

However,  the  CEM  program  office  continues  to  purchase  the 
warranty.  The  success  of  the  CEM  has  benefited  the  SFW 
system  because  the  systems  are  similar  and  have  some  common 
components . 

Redundancy  aspects  of  the  SFW  are  that  it  has  multiple 
identical  canisters  and  projectiles,  and  that  targets  are 
expected  to  be  hit  more  than  one  time  each.  Redundancy 
increases  system  reliability.  The  formula  developed  by  the 
system  program  office  to  calculate  system  reliability  seemed 
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appropriate  because  it  included  the  redundancy  aspects  of 
the  system.  The  series  reliability  formula  currently  in 
use,  per  direction  from  the  using  command,  does  not  project 
system  reliability  based  on  an  expected  mission  scenario,  as 
the  redundancy  formula  did.  The  two  formulas  result  in 
different  reliability  estimates,  and  in  turn  will  result  in 
different  projections  for  the  cost-effectiveness  of  the 
warranty. 

The  relationship  between  the  EPR  warranty  and  the 
defects  in  material  and  workmanship  and  design  and 
manufacturing  warranty  should  be  investigated.  Would  it  be 
possible  to  make  valid  warranty  claims  for  the  other  two 
parts  of  the  warranty,  1)  design  and  manufacturing,  and  2) 
defects  in  material  and  workmanship,  if  the  EPR  warranty 
were  waived?  It  seems  that  it  would  be  difficult  to  make 
claims  if  the  only  method  of  detecting  defects  and  faulty 
design  is  by  visual  inspection  only. 

The  following  are  recommendations  to  consider  for 
further  evaluation  of  the  warranty  effectiveness: 

1.  Consider  performing  a  cost-effectiveness 
sensitivity  analysis  using  the  reliability  formula 
that  considers  redundancy. 

2.  Determine  that  the  other  two  parts  of  the  warranty 
can  be  enforced  without  the  EPR  warranty  if  it 
were  waived. 

3.  Perform  an  analysis  to  compare  the  ”point-in-time" 
versus  the  cumulative  measurement  methods  used  to 


detect  a  breach  in  the  EPR  warranty,  relative  to 
the  expected  contractor  cost  for  the  warranty. 

The  potential  cost  and  benefit  differences  should 
then  be  analyzed.  The  possibility  exists  that  the 
warranty  could  be  cost-effective  if  an  alternate 
method  of  measurement  were  used  for  the  EPR 
warranty.  That  possibility  could  be  investigated. 

The  second  objective  of  this  research  question  was  to 
review  the  SFW's  and  other  dormant-  system's  CBAs  to 
determine  if  a  general  methodology  could  be  developed  to 
improve  the  CBAs. 

Several  CBAs  for  dormant  systems  were  examined,  and  the 
general  methodology  used  for  all  of  the  CBAs  included 
utilization  of  historical  data  and  projections  for  future 
costs  versus  expected  benefits.  Sensitivity  analysis  was 
performed  to  varying  degrees.  All  analyses  performed, 
except  for  the  Maverick  Missile,  also  included  life  cycle 
cost  with  and  without  a  warranty.  The  Maverick  Missile  CBA 
appears  to  be  the  weakest  for  the  following  reasons:  1}  it 
does  not  compare  life  cycle  cost  with  and  without  a 
warranty,  2)  the  only  cost-benefit  sensitivity  performed 
compares  a  four  and  five  percent  failure  rate  for  the  EPR 
warranty,  and  3)  the  cost  of  the  warranty  is  not  known 
(cost  is  included  in  the  missile  price).  The  Maverick 
Missile  warranty,  however,  is  the  only  one  that  shares  risk 
with  the  contractor  by  accepting  responsibility  for  the 
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first  four  percent  of  failures.  Risk  sharing  reduces  the 
warranty  cost. 

Appendix  C  illustrates  that,  for  the  warranty  CBA,  the 
systems  reviewed  use  many  different  cost  categories,  and 
only  a  few  categories  are  used  to  measure  benefits. 
Intangible  benefits  such  as  increased  mission  readiness  and 
increased  availability  were  not  included  in  any  of  the  CBAs. 
The  fact  that  benefits  were  omitted  tends  to  supports  the 
GAO's  findings  that  CBAs  are  inadequate. 

The  recommendation  is  to  tailor  the  CBAs  specifically 
to  correspond  with  the  warranty  contract.  Although  some 
similarity  exists  between  the  CBAs,  the  differences  in  the 
individual  warranties  do  not  suggest  a  universal  methodology 
should  be  developed.  Warranties  are  tailored  to  meet  the 
requirements  of  the  specific  system,  and  none  are  exactly 
alike.  The  remedies  specified  in  each  individual  warranty 
determines  the  factors  that  should  be  used  to  calculate 
costs  and  benefits.  The  SFW  system  has  a  sole-source 
contract.  An  acceptable  warranty  is  practically  impossible 
to  negotiate  in  a  sole-source  situation.  This  impediment 
should  be  a  consideration  when  planning  the  warranty 
strategy. 

Recommendations  for  Further  Research 

Dormant  weapon  system  present  a  unique  challenge  when 
developing  a  warranty  contract  and  deciding  how  to  enforce 
the  contract.  Although  a  limited  number  of  warranties  on 
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dormant  systems  currently  exist,  further  research  could  be 
performed  to  assess  warranty  feasibility  as  more  data 
becomes  available. 

Summary 

The  purpose  of  this  research  effort  was  to  identify  factors 
that  affect  the  CBA  for  the  Sensor  Fuzed  Weapon  (SFW)  System 
because  an  essential  performance  requirement  (EPR)  warranty 
waiver  was  under  consideration.  Defense  funding  has  become 
scarce,  and  the  government  must  spend  the  limited  dollars 
wisely.  A  CBA  should  provide  a  relevant  projection  of  cost- 
effectiveness  based  on  the  best  available  data.  The  SFW  and 
other  dormant  systems  present  difficulties  in  determining 
cost-effectiveness  because  the  systems  most  probably  will  be 
non-operating,  non-reusabl e ,  and  non-restorable.  The 
recommendations  for  the  SFW  program  office  were  1)  perform 
cost-effectiveness  sensitivity  analysis  using  initial 
reliability  formula,  2)  insure  warranty  remedies  could  be 
enforced  for  the  other  two  warranty  parts  if  the  EPR 
warranty  is  waived,  and  3)  review  the  method  of  measurement 
for  EPR  effectiveness  by  performing  a  CBA  comparing  the 
current  method  of  using  "points-in-time"  with  the  alternate 
method  of  using  a  cumulative  measurement.  The  SFW's  EPR 
warranty  will  become  effective  in  1996.  In  the  meantime, 
the  program  manager  can  examine  more  closely  the  feasibility 
of  the  EPR  warranty  and  the  warranty  in  total . 
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Appendix  A:  SFW  Sampling  Plan 
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shown  below  is  the  formula  used  to  calculate  the  system 
reliability  that  considers  redundancy: 


[  1  -  ( 1  -Rwd)  X 1 1  -  ( 1  -R^)  "“1 


A  graphical  view  of  the  reliability  tree  displaying 
system  redundancy  is  shown  below: 


Where  TMD  is  a  Tactical  Munitions  Dispenser, 
KMU  is  an  adapter  kit  (KMU-488), 

DV's  are  Delivery  Vehicles,  and 
Projectiles 
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Appendix  C;  Comparison  of  Cost-Benefit  Analysis  Inputs 


Peace- 

Cost^ _ keeper 

Acquisition 

Set  up  for  Operation  ✓ 

Preventative  Maint.  «/ 

Warranty  Cost  «/ 


Adminis  ration 
Egineering  Change 
Proposals 
Parts 
Materials 
Fuel  Leaks 
Cost  of  Money 
Testing 

Essential  Performance 
Requirements 
Design  and 

Manufacturing 
Materials  and 
Workmanship 
Facilities 
Training 
Packing  and 
Shipping 
Technical 
Manuals 

Support  Equipment 
Labor 


_ Systems _ 

ACM 

ACM  Prop.  Maverick  CEM  SFW 


✓  ✓ 


✓ 

✓ 

✓ 


✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 


✓  ✓ 
✓  ✓ 

✓ 


✓ 

✓  ✓ 


✓ 


Benefits _ 

Failure  Repair  y  ^  y  ^ 

Spare  Parts 

Repair  Shop  y 

Test  (Analysis  of 
Results) 

Good  Units 


y 

y 

y 


Systems  Identified: 

Peacekeeper  Missile  Propulsion  System 
Advanced  Cruise  Missile  (ACM)  System 
ACM  Propulsion  System 
Maverick  Missile  System 
Combined  Effects  Munition  System 
Sensor  Fuzed  Weapon  System 
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